Examination of validity, reliability, and interpretability of a self-reported questionnaire on Occupational Balance in Informal Caregivers (OBI-Care) – A Rasch analysis

Objectives Informal caregivers often experience a restriction in occupational balance. The self-reported questionnaire on Occupational Balance in Informal Caregivers (OBI-Care) is a measurement instrument to assess occupational balance in informal caregivers. Measurement properties of the German version of the OBI-Care had previously been assessed in parents of preterm infants exclusively. Thus, the aim of this study was to examine the measurement properties of the questionnaire in a mixed population of informal caregivers. Methods A psychometric study was conducted, applying a multicenter cross-sectional design. Measurement properties (construct validity, internal consistency, and interpretability) of each subscale of the German version of the OBI-Care were examined. Construct validity was explored by assessing dimensionality, item fit and overall fit to the Rasch model, and threshold ordering. Internal consistency was examined with inter-item correlations, item-total correlations, Cronbach’s alpha, and person separation index. Interpretability was assessed by inspecting floor and ceiling effects. Results A total of 196 informal caregivers, 171 (87.2%) female and 25 (12.8%) male participated in this study. Mean age of participants was 52.27 (±12.6) years. Subscale 1 was multidimensional, subscale 2 and subscale 3 were unidimensional. All items demonstrated item fit and overall fit to the Rasch model and displayed ordered thresholds. Cronbach’s Alpha and person separation index values were excellent for each subscale. There was no evidence of ceiling or floor effects. Conclusions We identified satisfying construct validity, internal consistency, and interpretability. Thus, the findings of this study support the application of the German version of the OBI-Care to assess occupational balance in informal caregivers.

Introduction Persons with an impairment or a disease, such as preterm infants or persons with dementia, rely on the support and care of informal caregivers. Informal caregivers are defined as close family members, relatives or friends that provide unpaid care [1,2]. Informal caregivers often experience physical and psychological burden, stress, and discomfort [1,[3][4][5][6][7]. Moreover, informal caregiving leads to a restriction of meaningful activities for oneself which affects one's occupational balance [5,8,9].
Occupational balance is defined as a subjective balance between meaningful activities in different life areas, such as self-care, leisure time or productivity. Meaningful activities are characterized by having a specific purpose to a person and include activities a person does, wants to or has to do [10].Occupational balance is of high importance due to its association with health and well-being [11][12][13][14]. Its direct and indirect effects on health and quality of life could recently be confirmed [14]. Previous studies identified restricted occupational balance in informal caregivers [5,8,9,[15][16][17][18][19][20] and the need for interventions to improve their occupational balance [5,8,9]. Additionally, maintaining or improving informal caregivers' occupational balance might have positive effects on their health and well-being [9,[20][21][22]. For example, a study reported an association between parents' occupational balance and health and well-being of the child they cared for [22]. Therefore, it is important to address and assess occupational balance in informal caregivers. Due to subjectivity of occupational balance, a self-evaluation of one's occupational balance is needed [23,24]. Self-reported outcome measures, such as caregiver-reported questionnaires are required for self-evaluation [25]. These outcome measures consider the perspectives of the persons concerned and thus generate outcomes that are meaningful to them [25][26][27][28]. Additionally, self-reported outcome measures can be completed regardless of location and without the assistance of health professionals and are therefore inexpensive [29].
Reliable and valid outcome measures are prerequisites to assess deviations of occupational balance of informal caregivers, to set occupational balance interventions and to measure the effectiveness of these interventions [25]. However, self-reported outcome measures must comply with defined measurement properties, such as construct validity, internal consistency, and interpretability, to generate reliable and valid measurement outcomes [25,30,31]. Construct validity ensures accordance among scores of the outcome measure and existing knowledge or hypothesis, internal consistency ensures interrelatedness among a scale's items and interpretability ensures assignment of qualitative meaning in clinical practice [25,30].
Traditionally, examination of measurement properties has been guided by classical test theory (CTT). However, CTT methods to examine measurement properties have limitations. Item response theory approaches, such as analyses with a Rasch model, have been found to show advantages over CCT [32][33][34][35]. The Rasch model defines the probability that a person will answer an item correctly, given a specified person ability and item difficulty. Thus, the Rasch model provides a powerful approach to determine the coherence between the construct to be measured, and the outcome measure [32- 34,36].
Measurement instruments on occupational balance exist. However, these measurement instruments were not specifically developed with and validated in a sample population of informal caregivers [37,38]. The self-reported questionnaire on Occupational Balance in Informal Caregivers (OBI-Care [37]) is a generic measurement instrument to assess occupational balance in informal caregivers [37]. It was specifically developed with parents of preterm infants, who are considered to be informal caregivers. A German version of the OBI-Care was developed first and subsequently translated into English, only the German version is validated. Previous analyses of the measurement properties of the German version in a sample of parents of preterm infants demonstrated construct validity and internal consistency [37]. However, measurement properties of the German version of the OBI-Care have not been examined in a mixed population of informal caregivers, such as caregivers of persons of different ages and diagnoses [37]. The exploration of its measurement properties in a mixed population of informal caregivers is required to ensure its generic applicability to assess occupational balance in a wider range of informal caregivers.
Thus, the aim of this study was to examine construct validity, internal consistency, and interpretability of the German version of the OBI-Care in a mixed population of informal caregivers.

Design
We conducted a psychometric study, applying a multicenter cross-sectional study design. Measurement properties of the German version of the OBI-Care were analyzed. Specifically, construct validity, internal consistency and interpretability were addressed. This study was part of a larger study, the Occupational Balance Project of Informal caregivers (TOPIC).

Data collection
From September 2016 to July 2020, numerous strategies were applied to recruit informal caregivers for this multicenter study in Austria. These were personal recruitment in participating centers and self-help groups as well as electronic recruitment through posts on social media (Table 1).
Informal caregivers of persons treated in one of the participating centers and informal caregivers of participating self-help groups were informed about study procedures verbally and in writing by the principal investigator, study assistants, health professionals, including therapists and nurses and self-help group leaders. Subsequently, potential participants were asked to participate in this study and to fill in the paper survey (personal recruitment).
Additionally, informal caregivers were invited electronically to participate in this study. Therefore, the principal investigator, study assistants and self-help group leaders shared written information and a video about study procedures as well as the electronic survey on social media and on the homepages of their institutions (electronic recruitment). Inclusion criteria were informal caregivers i) who provided informal care for family members, relatives, or friends at the time of participation and ii) with sufficient German reading and writing skills. Exclusion criteria were underaged (� 18 years old) informal caregivers. No action was taken to recruit a sample that is representative of the population of informal caregivers in Austria.
Sample size was defined according to recommendations for Rasch analyses, were ten observations (cases) for each item in each category are required, whereby observations do not have to be individual cases [39].
Data collection instruments. Participants filled in the paper (personal recruitment) or the electronic (electronic recruitment) survey, digitalized with the program Enterprise Feedback Suite Survey [40], of a set of self-reported questionnaires [41][42][43][44][45][46] including the German version of the OBI-Care and the following sociodemographic data relevant for this study: informal caregivers' sex, age, caring effort, caring activities and the number of persons to be cared for as well as sex and age of the persons to be cared for. The OBI-Care consists of 22 items. Each item has a five-choice response scale, ranging from 1, very satisfied to 5, very dissatisfied. Items are summarized in three subscales. Subscale 1 (occupational areas) asks for the satisfaction with the extent of one's activities. Subscale 2 (occupational characteristics) asks for the characteristics and effects of one's activities. Subscale 3 (occupational resilience) asks for the adaptability of one's activities ( Table 2). The subscales represent multidimensionality and

Data analyses
Data was entered in a data file and analyzed with the Statistical Package for the Social Sciences (SPSS [47]) and Rasch Unidimensional Measurement Model 2030 (RUMM 2030 [48]). SPSS was used for factor and correlation analyses, RUMM 2030 for analyses with a Rasch Model. Participants who did not fill in the OBI-Care completely were excluded from analyses. Analyses were conducted for each subscale of the OBI-Care. Alpha's (α) level of significance was set at 0.05. For multiple testing, Bonferroni adjustment was applied [49,50]. Descriptive analyses. Descriptive analyses were carried out to describe sociodemographic data of informal caregivers and persons to be cared. Descriptive analyses included the calculation of means and standard deviations for normal distributed data and medians and interquartile ranges for non-distributed data as well as frequencies and percentages.
Examination of psychometric properties. Dimensionality testing and different analyses with a Rasch model were conducted to assess construct validity [34,[51][52][53][54][55][56][57]. Dimensionality was examined by factor analyses. Therefore, principal component analysis was applied to extract components and their eigenvalues. Components with eigenvalues � 1 were interpreted as an independent factor. One identified factor was interpreted as unidimensionality of a scale, factors � 2 as multidimensionality. Subscales should be unidimensional to guarantee that the included items measure the same construct [25,52]. Furthermore, item fit residual statistics and item-trait interaction chi-square statistics were analyzed to determine item fit and overall fit to the Rasch model. Non-significant item fit residuals (-2.5 to 2.5) and a mean item fit residual close to zero with a standard deviation close to one demonstrate an item fit. Non-significant chi-square values with a total chi-square probability value greater than 0.05 indicate overall fit [34,49,51,56,58,59]. Moreover, threshold ordering, and the representation of response categories were examined by exploring threshold maps and threshold probability curves. Ordered thresholds indicate that the item's response categories operate appropriately [34,49,51,56].
Correlation analyses were conducted to assess internal consistency. These were inter-item correlations, item-total correlations, Cronbach's α and the person separation index (PSI [25,50]). Inter-item correlations between 0.2 and 0.5 and Cronbach's α between 0.70 and 0.90 display that items measure the same construct and their appropriate allocation to the scale. Interitem correlations > 0.7 indicate that the items measure almost the same and one of them might be deleted. Item-total correlations of � 0.3 and a PSI � 0.7 imply that the items discriminate between persons with different abilities [25,50].
Interpretability was examined by the inspection of floor and ceiling effects [25]. Floor and ceiling effects are displayed when a high proportion (determined as 15%) of the sample population achieves the lowest (nine points for subscale 1, seven points for subscale 2 and six points for subscale 3) or highest score (45 points for subscale 1, 35 points for subscale 2 and 30 points for subscale 3) of an outcome measure. Floor and ceiling effects pose a problem in clinical practice, since persons that already achieved the lowest or highest score at baseline, cannot show any deterioration or improvement at follow up [25,60].

Ethical considerations
The current study was approved by the ethics committee of Lower Austria. Participants confirmed their voluntarily participation by returning the paper survey or completing the electronic survey.

Participants
In total, 217 informal caregivers participated in this study. Twenty-one participants were excluded due to missing data. Finally, data of 196 informal caregivers were included for analyses, extracted from 107 (55%) electronic surveys and 89 (45%) paper surveys. Sociodemographic data of informal caregivers and persons to be cared for are presented in Table 3. Persons to be cared for had different health conditions and diagnoses, such as cerebral palsy, dementia, cancer, or diabetes.

Construct validity
Overall, all subscales of the OBI-Care demonstrated good construct validity. Factor analyses showed that subscale 1 consists of two factors with eigenvalues � 1. Thus, subscale 1 did not satisfy unidimensionality. Subscale 2 and subscale 3 consisted of one component with an eigenvalue � 1 each and therefore complied with unidimensionality (Table 4).
For all subscales, item fit residuals ranged between -2.5 and +2.5 and mean item fit residuals were close to zero with a standard deviation close to one. Chi square probability values for each subscale were greater than 0.05. Therefore, all values indicated item fit and overall fit to the Rasch model. Detailed results of Rasch analyses are provided in Table 5.
All items of each subscale showed ordered thresholds. Additionally, all response categories were represented (Fig 1).

Internal consistency
Except for three item pairs of subscale 3, all items satisfied criteria for inter-item correlations (< 0.70). Inter-item correlations for item 3a and item 3b, item 3c and item 3d as well as for   Table 6.

Interpretability
Exploration of floor and ceiling effects indicated interpretability. No significant floor and ceiling effects were found. For subscale 1, one (0.6%) person of the sample population achieved the lowest score (9) and the highest score (45). For subscale 2, none (0.0%) of the participants reached the lowest score (7) and two (1.1%) reached the highest score (35). For subscale 3, two (1.1%) persons scored the lowest score (6) and three (1.7%) the highest score (30).

Discussion
Within this study we examined psychometric properties of the German version of the OBI-Care in a sample population of informal caregivers. Construct validity and internal consistency of the German version of the OBI-Care have already been examined in one of our previous studies [37]. However, the results of both studies differ partly and to our knowledge this is the first time that measurement properties of the OBI-Care have been examined in another population of informal caregivers.
As part of construct validity, we identified multidimensionality of subscale 1. This result does not comply with results of our previous studies, where subscale 1 displayed to be unidimensional [37]. However, it should be pointed out that we interpreted components with eigenvalues � 1 as an independent factor. There is no uniform definition as to which value the eigenvalue has to exceed to be defined as a factor [25]. In other studies eigenvalues are defined as an independent factor from � 3 onwards [55,61]. Taking this definition into account, subscale 1 would consist of only one factor and be unidimensional. Additionally, to our knowledge the OBI-Care is the first occupational balance measure that considers multidimensionality of occupational balance in terms of measurement properties by using subscales [24,37]. Thus, further analyses on the subscales of the OBI-Care are warranted. Examination of internal consistency indicated that item 3a on changed chronology and item 3b on adapted time expenditure as well as item 3e on perpetuating occupations and item 3f on finding new occupations were statistically redundant. This result differs from our previous study [37]. It is possible that it is not important for informal caregivers which kind of meaningful occupations and in which order or amount of time they are performed, as long as the performance is possible. Inter-item correlations for item 3c on knowledge gathering and item 3d on skills acquisition indicated redundancy as well. Within our previous study we came to the same conclusion [37]. Thus, we believe that participants do not differ between  knowledge gathering and skills acquisition. Supplementing these items with an example might enhance comprehensibility of the items. Since occupational balance is a latent construct, it cannot be assessed directly [23]. Additionally, there is no consent how to assess occupational balance. In line with other existing occupational balance measures [62,63], items of the OBI-Care ask for satisfaction with manifest components of occupational balance. Another occupational balance measure asks for the ability to perform manifest components of occupational balance [23]. Lack of consensus on the conceptualization and dimensions of occupational balance [24,64] leads to inconsistent occupational balance measures and uncertainty how to measure occupational balance. Therefore, further studies on the conceptualization and dimensions of occupational balance are required. The examination of interpretability of the OBI-Care is novel and thus provides new findings on the application of the OBI-Care in clinical practice. However, by calculating floor and ceiling values we determined the OBI-Care's capability to measure the full range of occupational balance exclusively. Further explorations on interpretability, such as cut off values and minimal important change are recommended [25,37].
Previous studies indicate that caregivers' occupational balance and the engagement in meaningful activities might have an impact on caregivers' subjective health and wellbeing as well as on subjective health and well-being of the persons to be cared for [5,8,9,20,21]. Thus, it is recommended that health professionals, such as occupational therapists, support informal caregivers' engagement in meaningful activities and thereby strengthen their occupational balance.

Strengths and limitations
This study shows several strengths and limitations. Construct validity, internal consistency, and interpretability present essential components of psychometric properties. However, further studies are warranted to examine other psychometric properties, such as responsiveness [25]. The examination of psychometric properties using analyses with a Rasch model facilitates the identification of measurement inadequacies that might not be detected by classical test theory and thus provides a powerful alternative [32][33][34]. We examined psychometric properties in a mixed sample population of informal caregivers to ensure applicability independent of the caregivers. The examination of measurement properties might be replicated in diverse populations characterized by informal caregivers of people with specific diagnoses, such as dementia. We examined psychometric properties of the German version of the OBI-Care exclusively. Therefore, an examination of the existing English version of the OBI-Care [37] is required.
Additionally, the multicenter design and numerous recruitment strategies led to a high diversity of persons to be cared for. However, it should be noted that 87.2% of informal caregivers included in this study were female. Analyses within a sample with more male informal caregivers might differ. Differences in the burden perceived by female and male informal caregivers have been identified previously [65]. Additionally, occupational balance has been found to differ in women and men [66,67]. Moreover, our study supports findings of previous studies that informal care is still mainly provided by women [68,69] and thus indicate the consideration of gender specific research on informal caregivers. Furthermore, it has to be considered that the application of numerous recruitment strategies (personal and online recruitment) may have led to potential bias [70].

Conclusion
The German version of the OBI-Care demonstrates construct validity, internal consistency, and interpretability. Thus, the OBI-Care can be applied to measure occupational balance in informal caregivers and to assess effectiveness of occupational balance interventions for informal caregivers.